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Abstract 

The change of government in 2010 provoked a large structural change in the English education 
landscape. Unexpectedly, the new government offered primary schools the chance to have ‘the 
freedom and the power to take control of their own destiny’, with better performing schools given a 
green light to convert to become an academy school on a fast track. In England, schools that become 
academies have more freedom over many ways in which they operate, including the curriculum, staff 
pay, the length of the school day and the shape of the academic year. However, the change to allow 
primary school academisation has been controversial. In this paper, we study the effect for the first 
primary schools that became academies. While the international literature provides growing evidence 
on the effects of school autonomy in a variety of contexts, little is known about the effects of 
autonomy on primary schools (which are typically much smaller than secondary schools) and in 
contexts where the school is not deemed to be failing or disadvantaged. The key finding is that 
schools did change their modes of operation after the exogenous policy change, but at the primary 
phase of schooling, academisation did not lead to improved pupil performance. 
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1. Introduction 


Since 2010, the educational landscape in England has radically altered. Nearly two-thirds of 
secondary schools and a fifth of primary schools are now academies, which are schools that 
have been granted considerable operational autonomy by government. As Michael Gove, the 
Minister then responsible, put it schools have been ‘given the freedom and the power to take 
control of their own destiny’. 1 

Although academies have been around as a school improvement policy for 
underperforming secondary schools since 2002 (and studied by Eyles and Machin, 2015, and 
Eyles at ah, 2016a, 2016b), the programme was radically altered and expanded following the 
election of the new government in May 2010. It became a structure to which all schools were 
invited to aspire. Enabling legislation was rapidly put in place two months after the election 
of the new government. For the first time, and through a completely unexpected policy 
change, primary schools were invited to become academies, with better performing schools 
given priority to convert. The first batch of such schools converted in the school year 
beginning in September 2010. In this paper, we study the impact of primary school 
conversion, three to four years after academisation took place. 

This policy has been introduced in an international context where publicly-funded, 
autonomous schools have become a more familiar school improvement policy, most notably 
through charter schools in the US, free schools in Sweden and secondary academies in 
England. Evaluations of the fonner tend to find achievement gains associated with charter 
status and with the ‘injection’ of charter school features to public schools. 2 Evaluations of 

1 Department for Education (2013). Forward by Michael Gove MP. 

2 However, there is some controversy within the literature. Recent experimental evaluations of charters in or 
near particular US cities (Boston and New York) find positive impacts on educational achievement (see 
Abdulkadiroglu et al. 2011, 2014; Angrist et al. 2013, 2016; Dobbie and Fryer 201 1 ; Hoxby and Murarka 2009). 
Wider coverage non-experimental evaluations produce more mixed results (Center for Research on Education 
Outcomes, 2009). On the injection of charter school features to public schools in Houston, and their beneficial 
effects, see Fryer (2014). 


2 



the Swedish free schools find some short-tenn effects, but no evidence of medium to long- 
term effects (see Bolhmark and Lindahl, 2015). 

The policy studied here is different from many others in the literature in three 
respects: (1) it involves conversion of existing schools rather than the creation of new 
schools; 3 (2) it is about the voluntary conversion of more highly perfonning schools and not 
the forced conversion of failing schools. The former tend to have lower proportions of 
children from disadvantaged backgrounds; (3) we focus on effects for young children (aged 
11) who attend quite small schools (compared to secondary schools). Although there have 
been evaluations of elementary schools in the charter school context, this appears to be less 
often the subject of evaluation than middle and high schools. Furthermore, the policy being 
studied here was in no way anticipated by schools or parents, which gives us leverage to 
identify causal effects. 4 

In the English context, academy schools operate outside local authority control, 
having autonomy to operate in areas such as hiring and pay of teachers, schools admissions 
(subject to national rules), curriculum (subject to some conditions), and decisions about the 
length of the school day and tenn. Many of these areas fall within the sphere of process and 
personnel decisions which are claimed in some economics of education research to exert 
positive effects on student outcomes because of superior infonnation held by local decision 
makers (Hanushek and Woessmann, 2011). Indeed, the first secondary schools in England to 
become academies (in the early 2000s) did seem to deliver positive effects on student 
outcomes (Eyles and Machin, 2015). However, the context was one in which a couple of 


3 Although this is also the case for Clark (2009) or Eyles and Machin (2015) for English secondary schools and 
to Abdulkadiroglu et al. (2014) for schools in New Orleans. 

4 The proposal that the opportunity to become academies be extended to primary schools was first raised in the 
Conservative Manifesto that was published in April 2010 (one month before the election) and implemented 
soon afterwards. This came as a surprise at the time and, importantly for establishing causality, was well after 
initial schooling decisions were made by the cohorts considered in this analysis. 
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hundred (previously underperforming) secondary schools became academies. It is not 
necessarily the case that these positive effects carry through to better performing schools 
and/or to (much smaller) primary schools. 

Better performing schools have done well within existing structures (by definition) 
and small schools may not have the expertise to cope with the increased responsibility that 
goes along with academy status. For example, they can no longer rely on Local Authorities 
for finance and accountancy. They need to report company accounts (like private companies). 
Governors (who are unpaid) are classed as trustees or directors of the company sitting on 
boards (rather than on a committee). Heads are personally responsible to the governing body 
for upholding good accounting standards. Thus, the management and reporting requirements 
of becoming an academy are not to be taken lightly and small schools may not necessarily 
have the range of expertise within their organisation. Schools are given a £25,000 grant to 
support the conversion process. However, in recognition of some of the difficulty of taking 
on ‘back office’ functions and the potential for economies of scale, there is encouragement 
and incentives to convert in chains or undertake some post-conversion collaborative 
arrangement with other schools. 

In fact if the autonomy offered within the academies model was unambiguously 
advantageous for schools, one would imagine that all schools would want to become 
academies. However, the UK government have had to back out of a policy to force all schools 
in England to become academies by the end of 2022 because of fierce hostility to this by the 
educational establishment (although the government vision is now to encourage, but not 
make it compulsory, that all schools should become academies). 

Whether such radical upheaval is in the interests of students is an empirical question. 
Most schools yet to convert are primary schools, which represent the vast majority of schools 
in England (although they are much smaller than secondary schools). One might hypothesise 
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that schools which volunteered to convert to academy status early-on are those that were most 
amenable to academy status, anticipating positive benefits. If effects are not found for such 
schools, one might question whether it is such a good idea to extend it to schools that are less 
enthusiastic. 

To evaluate the impact of academy conversion, we first select a treatment and control 
group of schools. The treatment group consists of primary schools that converted to academy 
status between 2010 and 2012. The control group are those that converted, but after the end 
of our sample period (2015 and 2016). We show that these treatment and control groups have 
similar pre-trends in outcome variables. Further, we study performance effects for pupils who 
were already enrolled in the primary school prior to conversion (and who had completed their 
first major assessment at age 7) and who were affected by academy conversion for their 
remaining primary schooling, which finishes at age 1 1 with national tests in English and 
maths. This legacy enrolment strategy mirrors that used in Eyles and Machin (2015) in their 
evaluation of the first underperforming secondary schools to become academies in the early 
2000s. It also draws on Fryer (2014) who looks at the effect of injecting charter school 
practices into traditional public schools and Abdulkadiroglu et al. (2014) who study school 
takeovers in New Orleans, referring to pupils who stay in converting schools as ‘grand- 
fathered’ pupils. 

The importance of estimating effects for pupils who were already enrolled in the 
school prior to conversion is that student mobility post-conversion is potentially endogenous 
to the policy itself. For example, parents may be attracted by the idea of academy status and 
be more likely to enrol their students to newly converted primary schools. Exit from the 
school post-conversion might also be non-random (for example, if schools change policies 
in a way that is less attractive to certain students or their parents). However, a very strong 
first stage estimate (of the effect of academy status on the probability of pre-conversion 
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enrolment) suggests that our effect is estimated for the majority of eligible pupils in the 
school. 


The rest of the paper is structured as follows. In Section 2, we present a brief summary 
of related studies. In Section 3, we discuss primary education in England and how academies 
have been introduced. In Section 4, we describe the data and research strategy. In Section 5, 
we describe our main results and explore mechanisms in Section 6. We conclude in Section 
7. 

2. Related Literature - a Brief Summary 

This paper follows on most directly from Eyles and Machin (2015) who studied the effect of 
academy conversion on the first schools to become academies in England in the early 2000s. 5 
These were about 200 secondary schools, most of which were converted to academies 
because they were underperforming. As discussed in Eyles et al. (2016a), the characteristics 
of predecessor schools were very different to schools that became academies from 2010 
onwards (from which time there was also a massive expansion in the number of secondary 
academies, with over a thousand schools converting within a short period). The earlier 
programme very much focused on disadvantaged inner city schools, serving very deprived 
communities. However, within a year of the new government, schools that were allowed to 
convert to academy status were much higher up the national test score distribution. They also 
had much fewer students eligible to receive free school meals. As discussed below, this 
change in the characteristics of schools that became academies has to do with government 


5 An earlier English study that looked at the impact of schools changing status is by Clark (2009). He looked at 
whether students in secondary schools that became grant-maintained (in the late 1980s/early 1990s) performed 
better after their change in status. Although interpreted as a result of the increase in autonomy, the setting is 
very different to that considered here. 
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policy on what schools could be fast-tracked for conversion. Thus, the current paper applies 
a similar methodology, but in a completely different setting. 

The literature on charter schools in the US is relevant to our research because charter 
schools are also publicly funded schools that have operational autonomy from local and 
central government. However, most charter schools (like free schools in Sweden) are new 
schools rather than conversions. Most of the literature on charter schools that use a lottery 
design find positive effects on attainment for pupils who ‘win’ a place at a charter school 
relative to those that do not (e.g. Abdulkadiroglu et al, 2011, Angrist et ah, 2010, Angrist et 
al., 2013, Dobbie and Fryer, 2011, Dobbie and Fryer, 2013 and Hoxby et al. 2009) and on 
longer run outcomes such as college attendance (Angrist et al., 2016, and Dobbie and Fryer, 
2014). An exception is Gleason et al. (2010) who does not find evidence of average effects 
but does find evidence of improvements for disadvantaged students. However, literature 
using quasi-experimental designs has not found evidence of positive average results (Betts et 
al. 2006; Dobbie and Fryer, 2016; CREDO, 2009, 2013). 

Differences in the results between types of study might be for methodological reasons 
- for example, Dobbie and Fryer (2013) show that non-experimental methods lead to 
downward bias in the context of New York schools. It might also be because of different 
socio-economic contexts considered by different types of study. Most studies using a lottery 
design are based on charters serving disadvantaged children in urban areas and must also be 
based on over-subscribed schools (for the lottery design to work). Many of the studies using 
a lottery design are based on middle or high schools (exceptions include Dobbie and Fryer 
(2011, 2013), Hoxby et al. (2009), who also include elementary schools in their analysis). 
One way in which our study differs from much of this literature is that we are focusing on 
better-performing primary schools (not mainly serving disadvantaged students) that 
voluntarily convert to a structure (academies) which allows them more autonomy. 
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In respect of school conversion rather than creation of a new school, our study is 
related to the literature on schools that convert to charters, and the introduction of practices 
used in charters to US public schools. With regard to the fonner, Abdulkadiroglu et al. (2014) 
study the effect of conversions to charters in Boston and New Orleans. Being enrolled in a 
school the year prior to conversion makes a pupil eligible to be ‘grandfathered’ into the new 
school. Legacy enrolment is used as an IV for years of enrolment in a charter school and 
they find evidence of large positive effects on test scores. Fryer (2014) examines the 
introduction of practices used in charters to public schools in Houston. Although these 
practices are assigned randomly to schools, students can select out of the treatment sample. 
To mitigate this concern, Fryer (2014) uses assignment to treatment as an instrumental 
variable for actual attendance at a charter. He finds evidence for strong positive effects on 
pupil perfonnance because of mechanisms such as increased use of school time, the use of 
high dosage tutoring (administered by effective staff), the use of data in infonning instruction 
practices and a culture of high expectations. 

Much of this literature focuses on the effects of autonomy (in various guises) in 
communities that are very disadvantaged. The study setting of this paper offers an 
opportunity to evaluate whether such autonomy is advantageous in a more prosperous setting, 
where schools do not have intakes that are disproportionately from disadvantaged 
backgrounds. 

3. Primary Education in England and Academies Policy 

In England, children start school the September after they reach the age of 4. Most children 
attend a primary school up to age 11, after which they go to secondary school. 6 Schooling 

6 There is a small number of infant schools and middle schools in parts of the country. They are not included in 
this analysis unless they are ‘linked’, meaning that students at an infant school are prioritised for places at the 
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in England is organised into Key Stages. At the end of Key Stage 1 (age 7), pupils are 
assessed by their teachers in English, Science and Maths according to national guidelines. At 
the end of Key Stage (age 11), they undertake national tests in English and Maths. 7 These 
tests are used to construct Performance Tables for primary schools, which are publicly 
available. There is essentially no grade repetition within the system. 

Up until the introduction of academies in 2010, schooling had been organised at the 
local level into Local Education Authorities (LEAs). There are 152 LEAs in England and 
around 15,000 primary schools. The LEA’s main functions in relation to primary schools are 
in building and maintaining schools, allocating funding, providing support services (e.g. for 
children with special needs), and acting in an advisory role to the head teacher regarding 
school perfonnance and implementation of government initiatives. The LEA also appoints 
one or two representatives on to a school’s governing body - a group of parents, teachers and 
community representatives that provides governance to the school. LEAs typically also offer 
a number of administrative and management functions including training, personnel and 
financial services. Up until 2010, the majority of pupils (67%) attended ‘Community 
Schools’ in which LEAs are also the statutory employer of school staff, owner of the 
buildings and the authority that manages student admissions. Most other state primary 
schools are faith schools (which have greater autonomy from the LEA). Although parents 
can apply to send their child to any primary school (i.e. there are no strict catchment areas), 
popular schools are often oversubscribed and places are rationed according to a Schools 
Admissions Code, of which proximity to the school is a common criterion. 8 


junior school; in these cases, the proportion of infant school attendees switching to the linked junior school is 
very high and we treat the two linked schools as though they were one single school. 

7 Prior to 2010, students were also assessed in science. 

8 For more information about the operation of primary schools and local government prior to 2010, see Gibbons 
etal. (2011). 
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Academy schools were first introduced to English education in the early 2000s. In 
general, these academies were a small number of secondary schools that had been 
underperforming. When a new government was elected in May 2010, there were 203 
academies. Although ‘free schools’ and education reform were issues raised in the manifesto 
of the new government prior to their election, there was no mention of large-scale expansion 
of the academies programme. 9 The Academies Act of 2010 paved the way for schools to 
convert to academy status in much larger numbers. 10 

When a school becomes an academy, it is governed outside the Local Authority and 
is overseen and funded directly by central government. A school is run in many ways like a 
company, where governors are classed as trustees or directors and the principal/head teacher 
is the chief executive. Strong financial management and governance at the level of the 
individual academy are very important (National Audit Office, 2012), especially given that 
oversight is no longer provided by the Local Authority. Unlike Community Schools (i.e. most 
state primary schools), academies manage their own admissions. While they still have to 
adhere to the Schools Admissions Code, they may choose to run their admissions policy 
differently than in the past. Although academies are required to teach a broad and balanced 
curriculum, including English, maths, science and religious education, they are not legally 
required to use the national curriculum. They have the ability to set their own pay and 
conditions for staff and more freedom in their hiring decisions (e.g. they may hire unqualified 
teachers). As funding comes directly from the government rather than through the Local 
Authority, they have greater freedom on how to use the budget allocation (i.e. nothing is 


9 Free schools are completely new schools that can be set up by interested parties (e.g. parents). There are now 
about 400 free schools open or approved. 

10 Most new academies since 2010 are ‘converters’. However, some academies are sponsored (i.e. managed by 
a private team of independent co-sponsors) and these are schools that have been underperforming. We do not 
evaluate the effect of academisation on such schools (which are closer to the original New Labour academies, 
evaluated by Eyles and Machin, 2015). 
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withheld by the Local Authority to fund central services), though they also have the 
responsibility of organising payroll functions, insurance and accountancy functions in-house 
or by contracting this out. Academies also have the ability to change the length of the school 
day and the shape of the academic year (through tenn times). 

In the interests of minimising risk, the Department of Education adopted a phased 
approach to the criteria for schools wishing to convert (National Audit Office, 2012), 
prioritising better performing schools. A key component of this decision is the report by the 
Schools Inspectorate (Ofsted) that visits schools every 3-5 years and rates schools on a four 
point scale ranging from ‘outstanding’ to ‘inadequate’. At the time, about 20% of schools 
were rated as ‘outstanding’ and 50% as ‘good’. The government initially prioritised schools 
rated as outstanding and fast-tracked their applications for conversion. The first such schools 
were converted to academy status in September 2010. In November, this was extended to all 
good schools with outstanding features. There was also an opportunity for any school to 
convert (irrespective of Ofsted grade) if it joined an academy trust with an excellent school 
or an education partner with a strong record of improvement. Then in April 2011, the criteria 
was widened to include schools that were ‘performing well’, which included consideration 
of the last three years’ exam results, the latest Ofsted inspections, and financial management. 
Figure 1 shows the huge rise in the number of primary school academies in England between 
2010 and 2015. 

Schools are encouraged to convert in a chain or partnership because ‘this can enable 
schools to support one another once they are academies, share resources, experience and 
ideas. Such an approach is particularly valuable to small primary schools where working 
together allows economies of scale to be achieved’ (Department for Education, 2013). There 
are a variety of models that can be adopted to work collaboratively where the most formal is 
a multi-academy trust wherein all schools are governed by one trust and board of directors. 
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An alternative is an umbrella trust where academy schools work together while still retaining 
a certain level of independence and individuality. The third main model is a collaborative 
partnership, which is a looser arrangement between schools, with no shared trust or 
formalised governance structure. In 2013/14, about two-thirds of primary academies were in 
a multi-academy trust. 

4. Data and Methodology 

Data 

The National Pupil Database (NPD) is a census of all pupils in the state system in England. 
During the primary phase of education, this accounts for the vast majority of children. The 
NPD includes basic demographic details of pupils - such as ethnicity, deprivation (measured 
by whether they are eligible to receive free school meals), gender, and whether or not English 
is their first language. We also know the school attended and can link this infonnation to 
other school-level infonnation such as the date of conversion to an academy school and the 
date and grade of Ofsted inspections (which are publicly available data). The data is 
longitudinal and we can track students as they progress through the school system. 

As discussed in Section 2, the national curriculum in England is organised around 
‘Key Stages’, the first two undertaken in primary school (at ages 5-11) and the second two 
in secondary school (at ages 1 1 to 16). At the end of Key Stage, head teachers have a statutory 
duty to ensure that their teachers comply with all aspects of the Key Stage 1 assessment and 
reporting arrangements. The assessments are in reading, writing, speaking and listening, 
mathematics and science. Local Authorities (and other recognised bodies) are responsible 
for moderation of schools. Thus, although teachers make their own assessments of students 
(and therefore are susceptible to potential bias), there is a process in place to ensure that there 
is a meaningful assessment that is standardised over all of England. At age 7, students are 
given a ‘level’ (i.e. there is no test score as such). However, following standard practice, we 
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transform National Curriculum levels achieved in Key Stage 1 assessments into point scores 
using Department for Education point scales and control for these scores in our regressions. 

At the end of primary school (or the Key Stage 2 phase of education), pupils take 
national tests in English and maths, which are externally set and marked. These are the 
outcome variables used in our analysis. 

Methodology 

We are interested in evaluating the effect of academy conversion on pupil 
achievement in national tests taken by pupils at the end of primary school, i.e. Key Stage 2 
(KS2) when they are aged 1 1 . A conversion event E(t = c) is defined as occurring in the 
school year t that the school converts to an academy. 

The approach we adopt is to compare what happens before and after conversion 
compared to a set of control schools and so it is important to discuss the dimensions of the 
research design that enable us to interpret our estimates as causal. Firstly, we consider pupils 
who are already enrolled in the school and who have completed Teacher Assessments at Key 
Stage 1 (i.e. age 7) before the schools converts as an academy. This ensures that academy 
conversion is exogenous to the enrolment decision and enables us to control for a measure of 
prior attainment. Secondly, we limit the event study on pupil perfonnance to a maximum of 
four years post conversion, including the year of conversion itself. This is because there are 
4 remaining years of primary school after the Key Stage 1 assessment. Thus pupils affected 
by conversion in Year 2 of primary school (when KS1 assessments are taken) could have up 
to four post-conversion years of education in the academy (i.e. since their full enrolment in 
the school runs from E = c-l up to E = c+3). Similarly children affected by conversion when 
enrolled in the predecessor school in Year 3 could have up to three conversion years (to E = 
c+2 ), and so on for children in Years 4 and 5 in the predecessor school. 
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Incorporating these features into a research design enables us to estimate the causal 
impact of being in an academy. We use administrative data that follows pupils through their 
school careers to estimate the impact on Key Stage 2 performance by means of the following 
value added equation: 

J 

KS2 ist = a s + a t + 0 1 A ist *I(E>t=c)+ ^ + cp^Sl^-P v list (1) 

j=i 

where X denotes a set of control variables and where we restrict analysis to pupils enrolled 
in the pre-conversion school. However, not all pupils who end up taking their KS2 test at a 
school that becomes an academy (i.e. Ai S t= 1) were enrolled at the school pre-conversion. 
Conversely, some students who were initially enrolled in a school that converted to an 
academy (ITTist = 1) leave the school and take their KS2 tests elsewhere. Thus ordinary least 
squares estimates of 9i from (1) will not reflect a causal estimate. In the Appendix, we 
document the structure of the ITT groups, presenting the sample accounting used in our 
research design. 

Denoting the variable indicating treatment by an academy conversion as 
Z. = A. *I(E > t = c), the approach we adopt accounts for selection into and out of 

treatment by using intention to treat status (ITTist) as an instrument for Z; s t, to estimate a local 
average treatment effect (LATE) in the following set-up: 

J 

Zj St = a s + a t + e 2 ITT jst *I(E>t=c)+ ^ TT 2j X jist + cp 2 KSl jst + v 2jst (2) 

j=i 

J 

KS2 jst = a s + a t + 0 3 ITT ist *I(E>t=c)+ ^ 'rr 3j X jist + cp 3 KSl jst + v 3jst (3) 

i=i 

In the first stage, (2), estimates of 02 show the proportion of the ITT group that stay in the 
academy and take KS2 tests there. Equation (3) is the reduced form regression of KS2 on the 
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instrument. The instrumental variable (IV) estimate is the ratio of the reduced fonn 


coefficient to the first stage coefficient, 03 / 02 . 

Extending this IV setting to the event study framework we are able to estimate 
separate estimates for the four years from conversion onwards (E = c to c+3) using four 
instruments for whether a pupil is ITT for event year c, event year c+1 and so on. 1 1 
Comparison Schools 

A naive comparison between primary academies and all other state-maintained 
schools is likely to suffer from significant selection bias, since (as discussed above) 
conversion to an academy was done on a voluntary basis and better-performing schools were 
prioritised. 12 One might expect schools seeking to become academies to have common 
unobservable characteristics such as having a school ethos more in line with the academy 
model. To account for this we use pupils attending future converters - schools that convert in 
the 2014/15 and 2015/16 academic years - as a control group in a difference-in-differences 
setting. As we have two waves of treatment (i.e. schools converting in 2010/11 and 2011/12), 
we use a slightly different sample as a control group for each. In our control group for the 
first wave, we restrict attention to pupils who were in schools that convert in 2014/15 and 
2015/16 but would have been ITT had those schools converted in 20 10/201 1. We do the same 
for the second wave (but use 2011/2012 to define ITT) and pool the estimates together. For 
all pre-conversion years the control groups are identical - they are Y ear 6 pupils who sit their 
KS2 assessments in schools that converted to academy status in 2014/15 or 2015/16. 


11 Formally, an individual enrolled in a treatment school in event year c-i and academic year group k is, for 
instance, intention to treat for c+1 if c-i + (6-k) is equal to c+1, where 6 is the academic year group in which 
KS2 exams are sat. The binary instrument in equation (4) is equal to 1 only if any one of the four instruments 
used for the event study equals 1 . 

12 In other words, our instrument is only assumed to satisfy the exclusion restriction conditional on pupils being 
in a well-defined sub-sample of the population. 
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This approach is legitimised in the empirical findings we describe below where there 
are no differential pre-conversion trends in the same school years. Thus the data structure we 
use is a balanced panel of schools for the school years 2006/07 to 2013/14 with repeated 
cross-sections of Year 6 pupils. 

5. Results 

Balancing Tests 

Table la shows the extent to which treatment and control groups are balanced at baseline 
(2006/07) for the full sample of treatment and control schools. There is a significant 
difference with respect to KS2 and KS 1 scores prior to the policy, with treatment schools 
being better performing. Treatment schools also have a slightly lower proportion of pupils 
eligible to receive free school meals. This is not so surprising as the government prioritised 
better perfonning schools for conversion to academy status (as discussed above). If we look 
at the position of schools in the distribution of all schools in 2010, we see that treatment 
schools were higher up the distribution than control schools. Treatment schools were at the 
64 th and 68 th percentile in KS2 English and maths respectively. Control schools were at the 
52 nd and 5 1 st percentile in English and maths respectively. 

However, treatment and control schools are more balanced when we look within 
school inspection grades (as defined by Ofsted in 2007-10). In fact, as Table lb shows, within 
each category there are no statistically significant differences at baseline between treatment 
and control schools for any baseline characteristic. Furthennore, they come from similar parts 
of the KS2 test score distribution in 2010. Thus, we estimate regressions for schools within 
each Ofsted grade, as well as for the pooled sample. 13 

13 We put schools with satisfactory and unsatisfactory grades together as there are so few of the latter. When 
estimating the pooled regressions over all schools, all variables are interacted with Ofsted grade. 
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Main Results 


Table 2 shows estimates of the ITT and 2SLS specifications, including all controls. 
Coefficients are shown by wave and for the pooled sample. Columns 1-6 show estimates 
when the treatment is whether the school converts to academy status. Columns 7-12 show 
estimates for years of exposure. The first stage estimates are 0.91-0.93 in all cases, showing 
that the vast majority of legacy enrolled pupils stay in the school to take their KS2 exams. 
The regression estimates show a precisely estimated zero effect. On average, academisation 
had no impact on pupil performance for up to 3-4 years of exposure. Table 3 shows effects 
within schools of a particular Ofsted grade (i.e. outstanding; good; and 
satisfactory/unsatisfactory). However, the coefficients tell a similar story. Pupils did not 
benefit from attending an academy school irrespective of their predecessor school’s initial 
Ofsted grade. 

One might be concerned about the research design being potentially contaminated by 
differential pre-policy trends. Figure 2 therefore shows an event study for pupils from 4 years 
prior to academy conversion to 3 years later. Effects are seen to be precisely defined and 
zero on average for every single year (although confidence intervals widen in c+3). There is 
no sign of pre -policy trends or a gradual improvement in results post-conversion. Figures 3 
and 4 show that this is also true for pupils in schools within each Ofsted grade. 

Table 4 shows estimates for pupils by discrete years of exposure, ranging from one 
to a maximum of four. Again, there is neither any sign of a positive effect nor any suggestion 
that benefits might be increasing with years of exposure. If anything, the opposite is the case, 
as the absolute values of the negative coefficients mostly get larger with more years of 
exposure. 

One might also wonder if there is much heterogeneity in the effect of the treatment 
within groups of schools. We have explored this by estimating a separate treatment effect for 
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each school within categories of Ofsted grade. The results are illustrated in Figure 6 for 
schools classified as outstanding, good and satisfactory/unsatisfactory. In each case, the 
distribution of estimated treatment effects is quite strongly concentrated around zero, 
especially for ‘good’ schools, in the middle chart, for which we have the highest number of 
observations. 

Falsification Test 

To do a falsification test, we alter the year in which each academy school converted to 
four years prior to when they actually did. This ensures that there is no overlap between those 
that are actually treated and those that receive this ‘fake’ treatment. We then estimate 
regressions for 2003-2010 rather than 2007-2014 (i.e. the sample used in our main analysis). 
All the treatment and ITT variables are defined for the ‘fake’ conversion year so that pupils 
who attend Wave 1 and Wave 2 treatment schools are considered as ‘treated’ after 2007 and 
2008 respectively. Results are shown in Figure 5 and show that all estimated effects are zero. 
Thus, this placebo test offers additional evidence of there being no differential trend in 
treatment and control groups in the pre-policy period. 

6. Did Academies Change Their Modes of Operation? 

The empirical evidence presented in the previous section makes it clear that we can harness 
no evidence of pupil improvements in primary schools that became academies. This finding 
of a zero effect might arise from schools not having changed at all or from schools changing 
in ways that did not benefit students. In this section, we therefore consider three aspects of 
possible change. The first is whether primary schools took up the option to exercise the many 
academy freedoms that became available to them from increased autonomy post-conversion. 
The second is whether they changed their patterns of expenditure and the third is whether 
they changed their staffing and management structures. 
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Use of Academy Freedoms 

There have been various largely descriptive investigations into whether schools have 
used their academy freedoms (e.g. Academies Commission, 2013; Cirin, 2014). The 
Academies Commission (2013) concluded that take-up of freedoms had been ‘piecemeal 
rather than comprehensive’, in part because changes take time to be implemented and in 
practice require consultation (e.g. changing the length of the school day). Other hypotheses 
with some anecdotal support included potential risk aversion and insufficient skills of school 
leaders and that not all schools saw the need to innovate. Surveys of recent converters by 
Bassett et al. (2012) and Cirin (2014) found that financial motives were important in the 
decision to convert. In the former study, over 75% of respondents cited it as one of their 
reasons for converting and two-fifths as their primary reason. Cirin (2014) found that the 
desire ‘to gain greater freedom to use funding as you see fit’ was the most commonly cited 
reason for conversion (cited by 83% of respondents). 

Importantly for our analysis, Cirin (2014) reports change resulting from the exercise 
of academy freedoms to be more common in secondary than in primary schools. This is 
illustrated in Table 5, taken from his survey of 720 academies which were open on 1 May 
2013. This shows that for most categories, fewer than half of primary schools had made the 
changes suggested. In every case, the proportion of primary schools having made a particular 
change is smaller than the proportion of secondary schools making that change. In some areas 
(e.g. increasing the length of the school day; changing the length of school tenns and hiring 
teachers without qualified teacher status), there are very few schools reporting having made 
a change. At the same time, this Table shows that quite a lot of primary academies did make 
some changes. Furthermore, Cirin (2014) reports that two-thirds of academies believe that 
the changes they made have improved attaimnent. 

Changes in Patterns of Expenditure 
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The academy freedom studies suggest that the financial motive to convert was 
important. In Table 6, we therefore look at income and expenditure before and after 
conversion for our treatment and control schools using administrative data on school income 
and expenditure. There are some data issues that need to be highlighted upfront before 
discussing these numbers. First, the timing of reporting changed after conversion, with 
academies reporting in the September-August school year as opposed to the April-March 
financial year. The latter is in line with local authority financial statements and was the 
practice in schools before they converted to become an academy and in control schools 
throughout the period of our analysis. Secondly, academies opening between March and 
August have the option of filing a return that exceeds 12 months, but is less than 18 months. 
For some years we have information on whether schools take up this option and find that few 
do. However, the schools that do take up this option do so in a predictable way: for example, 
August converters file a 13 month return, July converters file a 14 month return and so on. 
We are able to use this information to impute expenditure and income data into a 12 month 
pro rata number, for those schools opening between March and August, when we do not 
know the true length of the return. 14 Throughout, however, we also show results for the sub- 
sample that report 12 month finances, with few differences emerging. Finally, as our main 
focus, we look at expenditure shares to see if academies alter their modes of spending. 
Considering shares is not affected at all by varying reporting periods as the number of months 
in the return is the same for total incomes/expenditures and their components. 

In columns (1) and (4) of Table 6, we see that the income and expenditure was similar 
for the treatment and control schools before conversion. For example, as shown in Panel A, 
total income in all treatment and control schools was £3,917 and £4,138 per pupil 


14 Further details are given in the Appendix. 
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respectively. Total expenditure was £3,883 and £4,107 per pupil in treatment and control 
schools respectively. These pre-conversion numbers are even more closely aligned for the 
comparisons within the Outstanding, Good and Satisfactory/Unsatisfactory groups of 
schools, shown in Panels B to D of the Table. 

It is evident, however, that the converting primary schools both got more money and 
spent more money post-conversion. The Table also shows the income and expenditure per 
pupil after conversion and is converted into a difference-in-difference estimate in the final 
column of the Table. This shows significant income and expenditure gaps arising after 
conversion relative to what happened in the control schools at the same time. The differences 
in total income and expenditure are estimated as £624 and £527 per pupil per year. The 
increases are very clearly driven by the relative increase in grant income. This is what one 
would expect since because of academy conversion, schools received their funding directly 
from the government. Formerly, about 10% of their income would have been top-sliced by 
the Local Authority for central services. A similar qualitative pattern is shown for schools 
classified as Outstanding, Good and Satisfactory/Unsatisfactory, but with higher income and 
expenditure shown for the latter schools most likely reflecting a higher proportion of 
disadvantaged students in this group. 

In Table 7, we show the change in categories of expenditure per pupil before and after 
conversion. There are two Panels, the upper one for the full sample, and the lower one for 
the sample of schools that report 12 month accounts. They look very similar and show that 
the bulk of the relative increase in per pupil expenditure is being used for non-teaching staff 
and running costs of the school. 

Table 8 shows expenditure shares, again for the full sample in the upper Panel and 
for 12 month return academies only in the lower panel. They are extremely similar and make 
it very clear that even though primary academies spent more within each category after 
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conversion (relative to control schools), the relative share spent on ‘frontline services’ such 
as teaching staff declined (with a small or negligible change in learning and ICT resources) 
whereas the relative share spent on administrative costs (i.e. non-teaching staff and other 
running costs) increased. This is true for schools of all Ofsted categories. 

In Figure 7, we plot estimated treatment effects (for each school) against the change 
in grant funding within those schools. We do this separately for each group of schools 
classified as Outstanding, Good and Satisfactory/Unsatisfactory. For each group, there is no 
obvious relationship between the estimated effects of academy conversion (on pupil 
attainment) and the change in grant funding. Given that the bulk of the financial gain from 
academy conversion was subsumed into activities formerly undertaken by the Local 
Education Authority, this is not very surprising. 15 
Changes in Leadership 

Eyles and Machin (2015) found change in leadership to be an important mechanism 
behind the positive effects found for conversion of early secondary Academies. In the upper 
panel of Table 9, we consider whether this mechanism is at work for primary academies. We 
estimate a difference-in-difference regression and show the effect of academy conversion on 
the probability of a change in the head teacher. We estimate these regressions for all schools 
(column 1) and then within each category of school (Outstanding, Good, 
Satisfactory/Unsatisfactory). In all cases, the estimates are small and not statistically different 
from zero. 

Changes in Teaching Staff 

The lower panel of Table 9 shows difference-in-difference estimates of staff 
turnover; turnover is measured as being the number of the previous year’s staff who are no 

15 In contrast, Holmlund et al (2010) and Gibbons et al. (2011) find positive effects from general increases in 
pupil expenditure in primary schools in England. 
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longer at the school weighted by the total number of staff in the previous year. The estimates 
in Table 9 measure turnover amongst fixed term and permanent staff, although the results are 
statistically indistinguishable when part-time staff are included. Similarly to head teacher 
turnover, there is no evidence of primary academies changing their staff differentially from 
non-academies upon conversion. 

Summary 

Taken together, these findings suggest that primary schools did change some aspects 
of their operation and expenditure after becoming academies even though there was not much 
change in the school personnel. However, they did not change radically post-conversion in 
any way likely to directly improve pupil perfonnance. Although extra resources were made 
available to them, the changes they actually made were mainly to affect their administrative 
functioning and day to day operations. 

7. Conclusion 

The English government has radically restructured its school system under an assumption 
that academisation delivers benefits to schools and students. We study the totally unexpected 
change in direction that occurred in 2010 when policy was changed to let primary schools 
become academies. We consider the first primary schools to become academies in England 
(between 2010 and 2012) and find no evidence of pupil performance improvements resulting 
from conversion. 

How should we interpret a zero effect in the light of evidence showing positive effects 
of autonomy in other contexts? One reason is that schools that converted were already doing 
well within the system and simply did not require additional autonomy in order to thrive and 
therefore did not make substantive changes. Another reason may be that these schools are 
not serving disadvantaged communities and therefore not making changes that will make a 
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difference to average or higher-performing students. In existing research, much of the 
positive effects of autonomous schools have been shown for disadvantaged students and not 
so much for advantaged students. While there was scope to improve achievement within these 
schools (i.e. on average they were in the 64 th -68 th percentile of national test scores prior to 
conversion), it may be that changes introduced as a result of school autonomy simply do not 
benefit such students at the margin. However, given the survey evidence reported above and 
our research into how additional income was used by schools, it would appear that many of 
these schools did not make changes that affect ‘frontline services’ (as opposed to 
administrative roles). Finally, one of the key models for some successful urban charters in 
the US and some secondary schools in England 16 - an effective discipline approach for 
academies and the No Excuses model of charters - is simply not relevant to the age range of 
children enrolled in English primary schools. In the light of all these factors, it is not 
surprising that there has been no effect on pupil perfonnance. 

One might argue that if academisation has no effect on pupil performance, this could 
still be a reasonable public policy if there are other reasons for why this might be beneficial 
- for example, if school leaders can more easily make changes that might benefit students (or 
their parents) and staff. However, the process of restructuring schools in England in this way 
seems excessively costly if there are no gains for students. Furthermore, risks are also posed 
by an increasing number of schools becoming academies. For example, they are no longer 
regularly monitored at the local level. Problems might not therefore come to light unless they 
are flagged up by an Ofsted inspection, which are not regular events. There are potential 
negative spill-overs on other schools if opting out of Local Authority control undennines 
services that the Local Authority is able to provide to other schools in the same geographic 

16 Probably the best known of the latter is Hackney's Haggerston School which has fully utilised an effective 
discipline and good behaviour approach in its successful rise up the KS4 achievement distribution, despite 
having a relatively disadvantaged pupil intake. 
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area (e.g. child psychologists to support children with special needs in many schools). On the 
other hand, it might be the case that academisation has beneficial effects on certain groups of 
students within these schools, or that effects are positive if schools work together (e.g. in the 
context of multi-academy trusts). These are some of the issues that need to be investigated in 
future research. 
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Figure 1: Number of Primary School Academies in England 
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Figure 2: Event Study Estimates, Pre- and Post- Academy Conversion 
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Figure 3: Event Study Estimates by (Pre-Intervention) Ofsted Grade, Maths 
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Figure 4: Event Study Estimates by (Pre-Intervention) Ofsted Grade, English 
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Figure 5: Falsification Test 
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Figure 6: Heterogeneous Treatment Effects 
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Figure 7: Estimated Treatment Effects and Changes in Grant Income 
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Table la: Baseline Characteristics: Pooled Sample and by Treatment Wave 



All treatment and control schools 

Wave 1 treatment v comparison 

Wave 2 treatment v comparison 


Treatment 

Control 

p-value 

Treatment 

Control 

p-value 

Treatment 

Control 

p-value 

Number of schools 

270 

395 


46 

395 


224 

395 


English is first language 

0.92 

0.93 

0.543 

0.91 

0.93 

0.361 

0.93 

0.93 

0.763 

White British 

0.83 

0.85 

0.347 

0.83 

0.85 

0.577 

0.83 

0.85 

0.401 

Eligible to receive free school meals 

0.10 

0.12 

0.049 

0.13 

0.12 

0.644 

0.09 

0.12 

0.013 

Male 

0.51 

0.51 

0.577 

0.51 

0.51 

0.727 

0.51 

0.51 

0.608 

KS2 English 

0.07 

-0.06 

0.000 

-0.04 

-0.06 

0.698 

0.10 

-0.06 

0.000 

KS2 Maths 

0.09 

-0.08 

0.000 

0.03 

-0.08 

0.144 

0.10 

-0.08 

0.000 

KS1 English 

0.61 

-0.05 

0.000 

-0.07 

-0.05 

0.823 

0.09 

-0.05 

0.000 

KS1 Maths 

0.05 

-0.04 

0.002 

-0.07 

-0.04 

0.642 

0.07 

-0.04 

0.000 



Table lb: 

Treatment v Control by (Pre-Intervention) Ofsted Grade 




Outstanding schools 


Good schools 



Satisfactory/unsatisfactory schools 


Treatment 

Control 

p-value 

Treatment 

Control 

p-value 

Treatment 

Control 

p-value 

Number of schools 

108 

51 


115 

188 


47 

156 


English is first language 

0.91 

0.91 

0.865 

0.94 

0.94 

0.875 

0.94 

0.93 

0.766 

White British 

0.81 

0.83 

0.667 

0.85 

0.86 

0.670 

0.84 

0.84 

0.994 

Eligible to receive free school meals 

0.09 

0.10 

0.795 

0.09 

0.10 

0.328 

0.14 

0.14 

0.821 

Male 

0.51 

0.49 

0.070 

0.51 

0.51 

0.688 

0.49 

0.51 

0.205 

KS2 English 

0.25 

0.24 

0.807 

0.01 

0.01 

0.971 

-0.26 

-0.26 

0.954 

KS2 Maths 

0.28 

0.23 

0.426 

0.04 

0.03 

0.649 

-0.33 

-0.30 

0.720 

KS1 English 

0.13 

0.06 

0.197 

0.04 

0.00 

0.299 

-0.09 

-0.15 

0.376 

KS1 Maths 

0.10 

0.02 

0.147 

0.06 

0.02 

0.288 

-0.14 

-0.13 

0.910 

Position in dist. of all schools 










(percentile) 










English 

75th 

79th 


60th 

65th 


39 th 

38th 


Maths 

72th 

73 th 


58th 

57 th 


38 th 

36th 



Notes: All variables are measured in the school year 2006/07. There are very few unsatisfactory schools in the sample (4 in the treatment and 18 in the controls). Hence we pool satisfactory 
and unsatisfactory schools together. All KS1 and KS2 scores are standardized to have mean zero and standard deviation of 1 (within the year and overall sample). Ofsted grades are measured 
prior to the policy. Since Ofsted inspect schools every 3-5 years (see Section 3), the grades here are the most recent grade between 2007 and 2010. For all schools, the position in the percentile 
distribution in 2010 for the treatment group is 64 th and 68 th for English and Maths respectively. For the control group, this is 52th and 51 st respectively. Treatment and control are more evenly 
matched within Ofsted grade. 
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Table 2: The Effect of Treatment on KS2 Test Scores (measured at age 11) 



ITT (Incidence) 

2SLS (Incidence) 

ITT (Exposure) 


2SLS (Exposure) 


(1) 

(2) 

( 3 ) 

( 4 ) 

( 5 ) 

(6) 

( 7 ) 

(8) 

( 9 ) 

(10) 

(11) 

(12) 


2011 

2012 

Pooled 

2011 

2012 

Pooled 

2011 

2012 

Pooled 

2011 

2012 

Pooled 

Maths 

-0.050 

0.001 

-0.011 

-0.057 

0.001 

-0.012 

-0.021 

0.004 

-0.005 

-0.023 

0.004 

-0.005 


(0.053) 

(0.022) 

(0.023) 

(0.058) 

(0.023) 

(0.025) 

(0.019) 

(0.009) 

(0.010) 

(0.021) 

(0.010) 

(0.011) 

English 

-0.048 

0.018 

0.000 

-0.053 

0.020 

0.000 

-0.023 

0.008 

-0.004 

-0.025 

0.008 

-0.004 


(0.053) 

(0.021) 

(0.022) 

(0.058) 

(0.022) 

(0.024) 

(0.017) 

(0.009) 

(0.009) 

(0.018) 

(0.010) 

(0.010) 

School fixed effects 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

Y ear dummies 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

Sample size 

126459 

187481 

313940 

126459 

187481 

313940 

126459 

187481 

313940 

126459 

187481 

313940 

Number of schools 

441 

619 

665 

441 

619 

665 

441 

619 

665 

441 

619 

665 

First stage 

0.915 

0.937 

0.932 




0.910 

0.937 

0.928 




coefficient 

(0.007) 

(0.003) 

(0.003) 




(0.008) 

(0.003) 

(0.004) 





Notes: Each cell is a coefficient estimated from a separate regression. Full controls are included (for gender, ethnicity, speaks English as first language, eligible for free schools meals, prior 
attainment, primary school). Standard errors are clustered at school level. 
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Table 3: The Effect of Treatment by (Pre-Intervention) OfSted Grade 




Outstanding 




Good 



Satisfactory/Unsatisfactory 



(1) 

(2) 

(3) 

(4) 

(5) 

(6) 

(7) 

(8) 

(9) 

(10) 

(ID 

(12) 


ITT 

Incidence 

2SLS 

Incidence 

ITT 

Exposure 

2SLS 

Exposure 

ITT 

Incidence 

2SLS ITT 

Incidence Exposure 

2SLS 

Exposure 

ITT 

Incidence 

2SLS 

Incidence 

ITT 

Exposure 

2SLS 

Exposure 

Maths 

-0.014 

-0.015 

-0.009 

-0.010 

0.008 

0.008 

0.002 

0.002 

-0.042 

-0.047 

-0.007 

-0.008 


(0.046) 

(0.049) 

(0.019) 

(0.020) 

(0.028) 

(0.030) 

(0.012) 

(0.013) 

(0.053) 

(0.058) 

(0.023) 

(0.025) 

English 

-0.014 

-0.015 

-0.011 

-0.012 

0.020 

0.021 

0.006 

0.006 

-0.018 

-0.020 

-0.004 

-0.006 


(0.044) 

(0.049) 

(0.017) 

(0.018) 

(0.027) 

(0.029) 

(0.011) 

(0.012) 

(0.049) 

(0.054) 

(0.009) 

(0.023) 

School fixed effects 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

Y ear dummies 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

Sample size 


73631 



143156 



97153 


Number of schools 



159 




303 




203 


First stage 

0.942 


0.938 


0.937 


0.934 


0.906 


0.898 

coefficient 

(0.004) 


(0.005) 


(0.004) 


(0.005) 


(0.007) 


(0.010) 


Notes: Each cell is a coefficient estimated from a separate regression. Full controls are included (for gender, ethnicity, speaks English as first language, eligible for free schools meals, prior 
attainment, primary school). Standard errors are clustered at school level. 
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Table 4: Effects by Year of Exposure 



All schools 

Outstanding 


Good 

Satisfactory/Unsatisfactory 


Maths 


English 

Maths 

English 

Maths 

English 

Maths 


English 

One year of exposure 

-0.015 


0.000 

-0.010 

-0.011 

0.021 

0.035 

-0.010 


-0.057 


(0.024) 


(0.024) 

(0.041) 

(0.042) 

(0.032) 

(0.033) 

(0.057) 


(0.056) 

Two years of exposure 

-0.004 


0.026 

0.015 

0.039 

-0.013 

0.010 

-0.007 


0.039 


(0.027) 


(0.025) 

(0.050) 

(0.049) 

(0.038) 

(0.036) 

(0.059) 


(0.051) 

Three years of exposure 

-0.012 


-0.011 

-0.031 

-0.042 

0.017 

0.018 

-0.037 


-0.020 


(0.030) 


(0.028) 

(0.059) 

(0.055) 

(0.038) 

(0.035) 

(0.070) 


(0.066) 

Four years of exposure 

-0.042 


-0.094 

-0.073 

-0.106 

-0.033 

-0.022 

0.075 


-0.121 


(0.069) 


(0.063) 

(0.088) 

(0.083) 

(0.099) 

(0.084) 

(0.207) 


(0.165) 

School fixed effects 

Yes 


Yes 

Yes 

Yes 

Yes 

Yes 

Yes 


Yes 

Y ear dummies 

Yes 


Yes 

Yes 

Yes 

Yes 

Yes 

Yes 


Yes 

Sample size 

313940 

73631 


143156 


97153 


Number of schools 


665 


159 


303 


203 


First stage coefficient on 1IT 


0.968 


0.972 


0.952 


0.972 


x one year of exposure 


(0.002) 

(0.003) 


(0.006) 


(0.003) 


First stage coefficient on 1IT 


0.931 


0.945 


0.905 


0.934 


x two years of exposure 


(0.004) 

(0.004) 


(0.008) 


(0.006) 


First stage coefficient on 1IT 


0.907 


0.924 


0.871 


0.912 


x three years of exposure 


(0.004) 

(0.005) 


(0.013) 


(0.006) 


First stage coefficient on 1IT 


0.852 


0.873 


0.800 


0.829 


x four years of exposure 


(0.011) 

(0.013) 


(0.013) 


(0.025) 



Notes: Full controls are included (for gender, ethnicity, speaks English as first language, eligible for free schools meals, prior attainment, primary 
school). Standard errors are clustered at school level. 
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Table 5: Use of Freedoms Since Becoming an Academy: Primary and Secondary Schools 



Secondary Schools 

Primary Schools 

Changed your pattern of capital expenditure 

63% 

54% 

Introduced savings in back-office functions 

62% 

54% 

Changed the performance management system for teachers 

63% 

49% 

Changed the curriculum you offer 

60% 

49% 

Changed school leadership 

51% 

43% 

Introduced or increased revenue-generating activities 

41% 

28% 

Hired teachers without qualified teacher status (ATS) 

23% 

8% 

Sought to attach pupils from a different geographical area 

14% 

5% 

Increased the length of the school day 

10% 

5% 

Changed the length of school terms 

6% 

2% 

Number of schools 

360 

334 


Source: Cirin (2014). Online survey of 720 academies that were open on 1 May 2013. 
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Table 6: Changes in School Income per Pupil and Expenditure per Pupil Before and After Academy Conversion 




Treatment Schools 



Control Schools 


Treatment - Control 


Before 

After 

Change 

Before 

After 

Change 

Difference-in-Difference 


(1) 

(2) 

(3) = (2) - (1) 

(4) 

(5) 

(6) = (5) - (4) 

(7) = (3) - (6) 

A. All Schools 

(256 Treatment, 393 Control) 

Total Income 

3917 

4738 

822 (105) 

4138 

4335 

197 (20) 

624 (107) 

Grant Income 

3732 

4478 

746 (104) 

3987 

4164 

177(19) 

569 (106) 

Other Income 

185 

261 

76 (17) 

151 

171 

21(4) 

56 (18) 

Total Expenditure 

3883 

4575 

693 (102) 

4107 

4273 

165 (20) 

527 (104) 

B. Outstanding 

(106 Treatment, 50 Control) 

Total Income 

3792 

4550 

759 (68) 

3887 

4181 

293 (80) 

465 (104) 

Grant Income 

3594 

4236 

642 (56) 

3744 

4009 

265 (81) 

377 (98) 

Other Income 

197 

314 

117(36) 

143 

171 

28 (9) 

89 (37) 

Total Expenditure 

3769 

4423 

654 (76) 

3820 

4061 

241 (39) 

413 (85) 

C. Good 

(110 Treatment, 187 Control) 

Total Income 

3893 

4632 

738 (74) 

3974 

4185 

211 (19) 

528 (76) 

Grant Income 

3709 

4407 

698 (70) 

3821 

4013 

193 (18) 

505 (72) 

Other Income 

185 

225 

40 (18) 

153 

171 

18(6) 

22 (18) 

Total Expenditure 

3849 

4472 

623 (79) 

3952 

4136 

184 (22) 

439 (82) 

D. Satisfactory /Unsatisfactory 
(40 Treatment, 156 Control) 

Total Income 

4314 

5532 

1218 (624) 

4417 

4567 

150 (38) 

1068 (616) 

Grant Income 

4161 

5314 

1153 (625) 

4267 

4397 

130 (35) 

1023 (617) 

Other Income 

153 

218 

65 (30) 

150 

170 

20 (8) 

45 (30) 

Total Expenditure 

4277 

5262 

985 (591) 

4387 

4506 

119 (41) 

866 (584) 


Notes: The sources for expenditure data are publicly available consistent financial reporting records for all state-maintained schools and academies financial benchmarking data for academy schools. The fonner 
are available at https://www.compare-school-performance.service.gov.uk/ and the latter can be accessed at https://www.gov.uk/government/collections/statistics-local-authoritv-school-fmance-data . For 
academies opening in April to August of the school year, incomes and expenditures in the first full year of conversion are appropriately scaled. 
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Table 7: Changes in Expenditure per Pupil Before and After Academy Conversion 


Schools Including Those With Adjustment For > 12 Month Accounts in First Full Conversion Year 



All Schools 

Outstanding 

Good 

Satisfactory/Unsatisfactory 


Pre-Change 

Mean 

Difference-in- 

Difference 

Pre-Change 

Mean 

Difference-in- 

Difference 

Pre-Change 

Mean 

Difference-in- 

Difference 

Pre-Change 

Mean 

Difference-in- 

Difference 


(1) 

(2) 

(3) 

(4) 

(5) 

(6) 

(7) 

(8) 

Total teaching staff 

2854 

142 (70) 

2674 

95 (56) 

2789 

57 (58) 

3049 

367 (393) 

Total non-teaching staff 

455 

167(18) 

407 

149 (24) 

435 

180 (24) 

512 

159 (68) 

Learning and ICT resources 

218 

21 (17) 

231 

-18 (34) 

210 

-4(17) 

222 

113 (66) 

Other running costs 

525 

196 (26) 

481 

187 (43) 

495 

207 (27) 

592 

227 (106) 

Number of treatment schools 

256 


106 


110 


40 


Number of control schools 

393 


50 


187 


156 



Schools With 12 Month Accounts in First Full Conversion Year 



All Schools 

Outstanding 

Good 

Satisfactory/Unsatisfactory 


Pre-Change 

Mean 

Difference-in- 

Difference 

Pre -Change 
Mean 

Difference-in- 

Difference 

Pre -Change 
Mean 

Difference-in- 

Difference 

Pre -Change 
Mean 

Difference-in- 

Difference 


(1) 

(2) 

(3) 

(4) 

(5) 

(6) 

(7) 

(8) 

Total teaching staff 

2873 

173 (123) 

2679 

120 (57) 

2797 

44 (38) 

3058 

505 (681) 

Total non-teaching staff 

460 

158 (26) 

412 

160 (32) 

434 

162 (30) 

514 

125 (108) 

Learning and ICT resources 

218 

0(23) 

238 

-81 (33) 

208 

2 (22) 

222 

103 (101) 

Other running costs 

531 

219(40) 

483 

145 (56) 

500 

230 (30) 

592 

379 (174) 

Number of treatment schools 

125 


47 


56 


22 


Number of control schools 

393 


50 


187 


156 
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Table 8: Changes in Expenditure per Pupil Shares Before and After Academy Conversion 


Schools Including Those With Adjustment For > 12 Month Accounts in First Full Conversion Year 



All Schools 

Outstanding 

Good 

Satisfactory/Unsatisfactory 


Pre-Change 

Mean 

Difference-in- 

Difference 

Pre-Change 

Mean 

Difference-in- 

Difference 

Pre-Change 

Mean 

Difference-in- 

Difference 

Pre-Change 

Mean 

Difference-in- 

Difference 


(1) 

(2) 

(3) 

(4) 

(5) 

(6) 

(7) 

(8) 

Total teaching staff 

0.708 

-0.056 (0.007) 

0.709 

-0.046 (0.01) 

0.713 

-0.060 (0.009) 

0.701 

-0.066 (0.024) 

Total non-teaching staff 

0.111 

0.025 (0.003) 

0.107 

0.023 (0.004) 

0.109 

0.029 (0.005) 

0.115 

0.016(0.01) 

Learning and ICT resources 

0.054 

0.000 (0.004) 

0.060 

-0.007 (0.006) 

0.053 

-0.003 (0.006) 

0.050 

0.011 (0.008) 

Other running costs 

0.128 

0.031 (0.005) 

0.125 

0.030 (0.007) 

0.125 

0.035 (0.006) 

0.133 

0.039 (0.018) 


Schools With 12 Month Accounts in First Full Conversion Year 



All Schools 

Outstanding 

Good 

Satisfactory/Unsatisfactory 


Pre-Change 

Mean 

Difference-in- 

Difference 

Pre-Change 

Mean 

Difference-in- 

Difference 

Pre-Change 

Mean 

Difference-in- 

Difference 

Pre-Change 

Mean 

Difference-in- 

Difference 


(1) 

(2) 

(3) 

(4) 

(5) 

(6) 

(7) 

(8) 

Total teaching staff 

0.708 

-0.047 (0.007) 

0.707 

-0.025 (0.011) 

0.713 

-0.057 (0.007) 

0.702 

-0.066 (0.028) 

Total non-teaching staff 

0.111 

0.021 (0.004) 

0.107 

0.024 (0.005) 

0.109 

0.025 (0.006) 

0.115 

0.001 (0.012) 

Learning and ICT resources 

0.053 

-0.008 (0.003) 

0.062 

-0.022 (0.006) 

0.052 

-0.006 (0.005) 

0.050 

0.007 (0.01) 

Other running costs 

0.128 

0.034 (0.006) 

0.124 

0.022 (0.009) 

0.126 

0.038 (0.006) 

0.133 

0.058 (0.022) 


Notes: The categories are only fully available for schools with a minimum number of observations in each (minor) cell. The above table only uses schools for which all categories are non-missing. 
Total teaching staff includes: teachers, educational support staff and supply teachers. Total non-teaching staff includes administrative and clerical staff (the major category); administrative supply; 
bought-in professional services; indirect employee expenses, development and training, staff related insurance. Other running costs include premises, energy, rates, insurance, and bought in 
professional services (curriculum) 
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Table 9: Change in Head Teachers and Teacher Turnover 




Pr[Change 

in Head Teacher] 



All Schools 

Outstanding 

Good 

Satisfactory/Unsatisfactory 


(1) 

(2) 

(3) 

(4) 

Academy x Post-Conversion (E = c to c+3) 

0.026 

0.034 

0.008 

-0.020 


(0.026) 

(0.048) 

(0.038) 

(0.070) 

School fixed effects 

Yes 

Yes 

Yes 

Yes 

Y ear dummies 

Yes 

Yes 

Yes 

Yes 

Sample size 

2815 

725 

1285 

805 

Number of schools 

563 

145 

257 

161 

Teacher Turnover 

Academy x Post-Conversion (E = c to c+3) 

-0.018 

-0.012 

-0.026 

0.006 


(0.009) 

(0.014) 

(0.015) 

(0.019) 

School fixed effects 

Yes 

Yes 

Yes 

Yes 

Year dummies 

Yes 

Yes 

Yes 

Yes 

Sample size 

3150 

745 

1430 

975 

Number of schools 

630 

149 

286 

195 


Notes: Based on data from the schools’ workforce census for the academic years 2009/10-2013/14. The subsample is the sample of schools who are observed in each of the years and report having 
a head-teacher in each year. School level regressions where E denotes event year and c is the year of conversion. Robust standard errors (clustered at the school level) are reported in parentheses. 
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Appendix 


This Appendix contains information on the way in which the sample of pupils and schools are 
selected for our analysis of primary academies and on issues related to the school income and 
expenditure data analysed in the paper. 

1). Sample Accounting Structure 

Table A1 describes the structure of the Intention to Treat (ITT) groups in both waves of 
academy conversions that we study. For Wave 1, our ITT pupils are those enrolled in the 
predecessor school in academic years 2-5 in 2009/10. For Wave 2, the ITT pupils are those 
enrolled in the predecessor school in academic years 3-5 in 2010/1 1. 

The first column of each table gives the number of ITT pupils in treatment and control schools. 
The next set of numbers give the number of ITT pupils who remain in the treatment/control 
group. For instance, in the Wave 1 table, those who are pre-enrolled as Year 2 students in 
2009/10 are expected to sit their KS2 tests in 2013/2014. Of the 2098 treatment students in the 
sample in 2009/10, 1824 are observed in the school four years later while 1 1483 of the control 
group students (out of 13402) remain in the control group four years later. Those who are pre- 
enrolled in later year groups have lower rates of attrition due to having less time to drop out 
prior to Year 6. 

In all the ITT regressions our outcome of interest is a dummy for being Intention to Treat 
interacted with a dummy for being pre-enrolled in a treatment school in the appropriate year 
group. 


Table Al: Sample Accounting Structure 


Wave 1 


Year group in 
2009/10 


2009/10 

2010/11 

2011/12 

2012/13 

2013/14 


Treatment 

2098 




1824 

Z 

Control 

13402 




11483 


Treatment 

2108 



1912 


J 

Control 

13028 



11609 


A 

Treatment 

2115 


1973 



4 

Control 

13055 


12176 



C 

Treatment 

2154 

2077 




J 

Control 

13460 

13026 





Wave 2 


Year group in 
2010/11 


2010/11 

2011/12 

2012/13 

2013/14 

O 

Treatment 

9629 



8827 

J 

Control 

13486 



1211 

A 

Treatment 

9407 


8841 


4 

Control 

13099 


12188 


c 

Treatment 

9553 

9288 



J 

Control 

13134 

12736 
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2). Income and Expenditure Data Sources 


The income and expenditure data come from two sources; first, data on income and expenditure 
for academy schools is from the publicly available (at the Department for Education website 17 ) 
benchmark accounts returns, required by the Department for Education, for all academy 
schools; second, data for maintained schools comes from consistent financial reports, which 
are also made publicly available, as part of the school perfonnance tables. 18 19 

While maintained schools and academies are both required to submit financial returns, so as to 
allow the public to benchmark schools spending against each other, the data collected is slightly 
different for academies and state schools. In particular, state schools file a return for the 
standard financial year (April to March) while academies file a return covering the academic 
year (September to August). Exemptions are also available for academies in terms of both the 
length of the return and whether or not a return must be filed. When schools convert between 
March and August of a given year they have the option to file a return that exceeds 12 months 
(but is less than 18 months); as well as this, schools that gain academy status within an 
academic year are not expected to file a return for that year. 

To compare changes in spending before and after conversion we use two years of data for each 
treated school, which are as close together as possible, but fall either side of the conversion 
date. Given the nature of when reports are filed, this entails a comparison of the 2009/10 and 
2011/12 reports for schools converting in the 2010/2011 academic year and a comparison of 
the 2010/11 and 2012/13 reports for schools converting in the 2011/2012 academic year. In 
both cases, we differences out differences in expenditure for the control schools in the same 
years and pool together these two separate estimates to give the results in tables 6-8. 

In order to correct the data for academies that convert between March and August (and so have 
the option to file a return of up to 18 months), we utilise data that is available for the 2013/14 
year that indicates the length of returns filed for academies converting between March and 
August 2013. From these data we calculate the percentage of primaries converting in March 
who file an 18 month return, the percentage of primaries converting in April who file a 17 
month return etc. down to the percentage of those converting in August who file a 13 month 
return. 20 We then apply the following weight to schools that convert in month X where X lies 
between 3 and 8. 


/9+month x \ 

1-Frac x + — J *Frac x 

where Frac x is the fraction of schools converting in month x that file a non 12-month return in 
year 2013/14. 


17 See < https://www.gov.uk/govemment/statistics/income-and-expenditure-in-academies-in-england-2Q14-to- 
2015> for 2014/15 data. 

18 <https://www.compare-school-performance.service.gov.uk/ > 

19 While the data are publicly available some variables are suppressed; for instance, teaching staff costs are 
suppressed for small schools for confidentiality reasons (it is also necessary to suppress other costs at random so 
as to make it impossible to impute teaching costs from total expenditure). We would like to thank Andrew Mellon 
and Robert Drake at the Department for Education for providing us with unsuppressed data for both academies 
and maintained schools. 

20 It should be noted that all schools filing a return exceeding 12 months do so in a predictable way i.e. they file a 
return that runs September to September plus any expenditures incurred between the month of conversion and 
September. 
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We also experimented with using the following weights for those converting between March 
and August: 


/9+month x \ 

y jg j *1 (expenditure., x <Frac x ) 

/9+month x \ 

^ ) *1 ( ex penditure s x > 1-Frac x ) 

Where expenditure sx is the percentile of the expenditure distribution at which school s, 
converting in month x, lies and 1() is the indicator function. Using either of these adjustments 
gives very similar results from using the uniform adjustment reported in the Tables. 

From Tables 6 to 8 in the main body of the paper, it can be seen that using this adjustment leads 
to very little differences from estimates using the subsample of academies that file a 12 month 
return. 


In Table 8 in the main body of the paper, we also break down expenditure into four broad 
subcategories. Table A2 shows a breakdown of the expenditures that are in each category. 21 

Table A2: Expenditure Categories 


Total Teaching Staff 

Total Non-Teaching Staff 

Learning and ICT 
Resources 

Other Running Costs 

Teaching Staff 

Cost of other staff 

Learning resources (not 

Premises staff 

Supply teaching staff 

Indirect employee 

ICT equipment) 

Building maintenance 

Supply teacher 

expenses 

ICT learning resources 

and improvement 

insurance 

Development and 


Grounds maintenance 

Agency supply 

training 


and improvement 

teaching staff 

Staff related insurance 


Cleaning and caretaking 

(minus) Receipts from 

Administrative and 


Water and sewerage 

supply teacher 

clerical staff 


Other occupation costs 

insurance claims 

Administrative supply 


Catering staff 

Education support staff 

Bought in professional 
services such as auditor 

costs 


Catering supplies 
(minus) Income from 
catering 

Energy 

Bought in professional 
services - curriculum 
Rates 

Exam fees 

Other insurance 
premiums 

Special facilities 


21 A detailed discussion of these categories is available at: 
<https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/423098/CFR_guidance_FINAL 
_150415.pdf> 
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